Emotion recognition-based artwork recommendation method and device, medium, and electronic apparatus

ABSTRACT

The present disclosure provides an emotion recognition-based artwork recommendation method and device. The method includes: obtaining a current biometric parameter of a user; determining a current emotion type of the user according to the current biometric parameter; selecting an image of an artwork corresponding to the current emotion type according to the current emotion type; and recommending an image of the artwork to the user by displaying the image of the artwork on the display screen.

CROSS-REFERENCE OF RELATED APPLICATIONS

The present application is a § 371 national phase application ofPCT/CN2018/100318 filed Aug. 14, 2018, which claims the benefit of andpriority to Chinese Patent Application No. 201710694834.0, entitled“EMOTION RECOGNITION-BASED ARTWORK RECOMMENDATION METHOD AND DEVICE,MEDIUM, AND ELECTRONIC APPARATUS”, filed on Aug. 15, 2017, the entirecontents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to computer technologies, and inparticular, to an emotion recognition-based artwork recommendationmethod, an emotion recognition-based artwork recommendation device, anda computer-readable storage medium and electronic device forimplementing the emotion recognition-based artwork recommendationmethod.

BACKGROUND

With the development of the social economy, quality of life isconstantly improving, and more and more people have begun to study art.With the development of digital image processing technologies, a trendtoward digitizing traditional paper-based images, paintings, and otherworks of art is becoming more and more popular.

At present, an electronic frame is slowly gaining popularity as a noveldisplay device for the works of art. The electronic frame can be placedin various places, such as home, art exhibition hall or office, etc. Theelectronic frame can be used to display the works of art in the form ofdigital images, which is loved by the majority of users.

However, the intelligence degree of the existing electronic frames islow. Therefore, it is necessary to provide a new technical solution toaddress one or more problems in the above solutions.

It should be noted that the information disclosed in the Backgroundsection above is only for enhancing the understanding of the backgroundof the present disclosure, and thus may include information that doesnot constitute prior art known to those of ordinary skill in the art.

SUMMARY

According to a first aspect of embodiments of the present disclosure, anemotion recognition-based artwork recommendation method is provided. Themethod may be applied in an electronic device having a display screen.The method includes:

obtaining a current biometric parameter of a user, and determining acurrent emotion type of the user according to the current biometricparameter; and

selecting an image of an artwork corresponding to the current emotiontype according to the determined current emotion type, and recommendingan image of the artwork to the user by displaying the image of theartwork on the display screen.

In an exemplary embodiment of the present disclosure, the method furtherincludes:

before obtaining the current biometric parameter of the user, performingclassification training processing on a plurality of biometricparameters of the user based on a deep learning algorithm to obtain anemotion classification model;

determining a current emotion type of the user according to the currentbiometric parameter includes:

performing processing on the current biometric parameter according tothe emotion classification model to determine the current emotion typeof the user

In an exemplary embodiment of the present disclosure, the plurality ofbiometric parameters include a plurality of biometric parameters indifferent classifications;

performing classification training processing on a plurality biometricparameters based on a deep learning algorithm to obtain an emotionclassification model includes:

performing the classification training processing on the plurality ofbiometric parameters belonging to the classification based on the deeplearning algorithm, so as to obtain a plurality of different emotionclassification sub-models correspondingly;

performing processing on the current biometric parameter according tothe emotion classification model to determine the current emotion typeof the user includes:

determining a classification of the current biometric parameter, andselecting a corresponding emotion classification sub-model to processthe current biometric parameter according to a classificationdetermination result, so as to determine the current emotion type of theuser.

In an exemplary embodiment of the present disclosure, the method furtherincludes:

when a plurality of different classifications of current biometricparameters of a user are obtained, recording durations in which the useris in different emotion types determined according to the biometricparameters in each classification;

selecting an image of an artwork corresponding to the current emotiontype and recommending the image of the artwork to the user by displayingthe image of the artwork on the display screen, which includes:

comparing the durations corresponding to the different emotion types,and selecting an image of an artwork corresponding to an emotion typewhich corresponds to a maximum one of the durations and recommending theimage of the artwork to the user by displaying the image of the artworkon the display screen.

In an exemplary embodiment of the present disclosure, the plurality ofbiometric parameters in different classifications include facial featureparameters and sound feature parameters.

In an exemplary embodiment of the present disclosure, the method furtherincludes:

recommending a selected image of the artwork to the user for displayaccording to a preset time frequency.

In an exemplary embodiment of the present disclosure, the method furtherincludes:

when no emotion type is determined, recommending a related image of anartwork to the user according to the user's historical data.

In an exemplary embodiment of the present disclosure, the deep learningalgorithm includes a convolutional neural network deep learningalgorithm.

In an exemplary embodiment of the present disclosure, the method furtherincludes:

after obtaining the current biometric parameter of the user, verifyingvalidity of an identity of the user according to the current biometricparameter.

According to a second aspect of the embodiments of the presentdisclosure, an electronic device is provided. The electronic deviceincludes:

a body; a display provided on the body and configured to display animage of an artwork; a biometric feature collection device provided onthe body and configured to collect a biometric feature parameter of auser; a processor (e.g., a hardware processor); a memory for storinginstructions executable by the processor; and wherein the processor isconfigured to:

select an image of an artwork corresponding to the current emotion typeaccording to the current emotion type, and recommend an image of theartwork; wherein the display is configured to display the image of theartwork.

In an exemplary embodiment of the present disclosure, the processor isfurther configured to:

before the current biometric parameter of the user is obtained, performclassification training processing on a plurality of biometricparameters of the user based on a deep learning algorithm to obtain anemotion classification model;

perform processing on the current biometric parameter according to theemotion classification model to determine the current emotion type ofthe user.

In an exemplary embodiment of the present disclosure, the plurality ofbiometric parameters include a plurality of biometric parameters indifferent classifications;

the processor is further configured to perform the classificationtraining processing on the plurality of biometric parameters belongingto the classification based on the deep learning algorithm, so as toobtain a plurality of different emotion classification sub-modelscorrespondingly;

the processor is further configured to determine a classification of thecurrent biometric parameter, and select a corresponding emotionclassification sub-model to process the current biometric parameteraccording to a classification determination result, so as to determinethe current emotion type of the user.

In an exemplary embodiment of the present disclosure, the processor isfurther configured to:

when a plurality of different classifications of current biometricparameters of a user are obtained, record durations in which the user isin different emotion types determined according to the biometricparameters in each classification;

the processor is further configured to compare the durationscorresponding to the different emotion types, and select an image of anartwork corresponding to an emotion type which corresponds to a maximumone of the durations and recommending the image of the artwork to theuser by displaying the image of the artwork on the display screen.

In an exemplary embodiment of the present disclosure, the processor isfurther configured to:

recommend a selected image of the artwork to the user for displayaccording to a preset time frequency.

In an exemplary embodiment of the present disclosure, the plurality ofbiometric parameters in different classifications include facial featureparameters and sound feature parameters.

In an exemplary embodiment of the present disclosure, the processor isfurther configured to:

when no emotion type is determined, recommend a related image of anartwork to the user according to the user's historical data.

In an exemplary embodiment of the present disclosure, the processor isfurther configured to:

verify validity of an identity of the user according to the currentbiometric parameter after the user's current biometric parameter isobtained.

According to a third aspect of the embodiments of the presentdisclosure, there is provided a non-transitory computer readable storagemedium having a computer program stored thereon. When executed by aprocessor, the program causes the process to implement steps of theemotion recognition-based artwork recommendation method according to oneof the above embodiments.

It should be noted that the above general description and the followingdetailed description are merely exemplary and explanatory and should notbe construed as limiting of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in the specificationand constitute a part of the specification, show exemplary embodimentsof the present disclosure. The drawings along with the specificationexplain the principles of the present disclosure. It is understood thatthe drawings in the following description show only some of theembodiments of the present disclosure, and other drawings may beobtained by those skilled in the art without departing from the drawingsdescribed herein.

FIG. 1 schematically shows a flowchart of an emotion recognition-basedartwork recommendation method according to an exemplary embodiment ofthe present disclosure.

FIG. 2 schematically shows another flowchart of an emotionrecognition-based artwork recommendation method according to anexemplary embodiment of the present disclosure.

FIG. 3 schematically shows still another flowchart of an emotionrecognition-based artwork recommendation method according to anexemplary embodiment of the present disclosure.

FIG. 4 schematically shows an image of an artwork according to anexemplary embodiment of the present disclosure.

FIG. 5 schematically shows another image of an artwork according to anexemplary embodiment of the present disclosure.

FIG. 6 schematically shows a schematic diagram of an emotionrecognition-based artwork recommendation device according to anexemplary embodiment of the present disclosure.

FIG. 7 schematically shows a schematic diagram of a computer readablestorage medium according to an exemplary embodiment of the presentdisclosure.

FIG. 8 schematically shows a schematic diagram of an electronic deviceaccording to an exemplary embodiment of the present disclosure.

DETAILED DESCRIPTION

Example embodiments will now be described more fully with reference tothe accompanying drawings. However, the embodiments can be implementedin a variety of forms and should not be construed as being limited tothe examples set forth herein; rather, these embodiments are provided sothat this disclosure will be more complete so as to convey the idea ofthe exemplary embodiments to those skilled in this art. The describedfeatures, structures, or characteristics in one or more embodiments maybe combined in any suitable manner.

In addition, the drawings are merely schematic representations of thepresent disclosure and are not necessarily drawn to scale. The samereference numerals in the drawings denote the same or similar parts, andthe repeated description thereof will be omitted. Some of the blockdiagrams shown in the figures are functional entities and do notnecessarily correspond to physically or logically separate entities.These functional entities may be implemented in software, or implementedin one or more hardware modules or integrated circuits, or implementedin different networks and/or processor devices and/or microcontrollerdevices.

Existing electronic frames are used only for displaying artworks, or canrecommend an artwork most viewed by a user according to a viewinghistory of the user. The intelligence level of existing electronicframes is low, and most of the recommendations are based on commercialvalue, which is not user-friendly. In view of the above problems, thepresent disclosure provides an emotion recognition-based artworkrecommendation method, an emotion recognition-based artworkrecommendation device, a computer-readable storage medium and electronicdevice for implementing the emotion recognition-based artworkrecommendation method.

An exemplary embodiment provides an emotion recognition-based artworkrecommendation method. The method can be applied to an electronic devicewith a display screen, such as an electronic frame. Embodiments of thepresent disclosure are described below taking the electronic frame as anexample, but the present disclosure is not limited to this. Theelectronic frame may include, for example, a rectangular frame, adisplay, a microphone, a camera, and the like (not shown). The displaymay be arranged in the frame and configured to display an image of anartwork such as a painting. The microphone and camera may be provided onthe edge(s) of the frame and configured to collect biometric parametersof a user, such as the user's sound features and facial image features,etc. Referring to FIG. 1, the method may include the following steps:

In step S101, a current biometric parameter of a user is obtained, and acurrent emotion type of the user is determined according to thebiometric parameter.

In step S102, an image of an artwork corresponding to the emotion typeis selected according to the determined current emotion type, and theimage of the artwork is recommended to the user for display.

Through the above emotion recognition-based artwork recommendationmethod, the technical solutions according to the embodiments of thepresent disclosure can intelligently determine the user's currentdifferent emotions and then recommend correspondingly different artworkimages, and thus the technical solutions according to embodiments of thepresent disclosure are highly intelligent and more user-friendly.Accordingly, the corresponding products such as the electronic frame,and so on can understand the user better and become friends in theuser's life.

Hereinafter, each step of the above methods according to the exemplaryembodiment will be described in more detail with reference to FIGS. 1 to5.

In step S101, a current biometric parameter of a user is obtained, and acurrent emotion type of the user is determined according to thebiometric parameter.

According to an exemplary embodiment, the biometric parameters mayinclude, but are not limited to, facial feature parameters, or soundfeature parameters. For example, the microphone and the camera on theelectronic frame may be used to collect the biometric parameters of theuser such as the sound feature parameters and the facial featureparameters, respectively. The emotion type may include, but is notlimited to, depression, anger, happiness, and the like. The differenttypes of emotions may be determined in advance by deep learning at leastone of the facial feature parameters and sound feature parameters of auser with a large number through the deep learning algorithm.

For example, the current facial feature of the user may be obtained, andthe current emotional type of the user, such as anger, may be determinedaccording to the facial feature.

In an exemplary embodiment of the present disclosure, the method mayfurther include: after the user's current biometric parameter isobtained, verifying validity of an identity of the user according to thebiometric parameter. For example, the current user's facial feature orsound feature is compared with the corresponding pre-stored facialfeature or sound feature to verify whether the current user's identityis valid. If the identity of the current user is valid, the subsequentstep of determining the emotion type is continued, and if the identityof the current user is not valid, the subsequent step is not performed,and re-verification of the identity of the user is performed.

As shown in FIG. 2, in order to set the emotion type in advance, in anexemplary embodiment of the present disclosure, the method may furtherinclude the following steps:

In step S201, before the current biometric parameter of the user isobtained, classification training processing is performed on a pluralityof biometric parameters of the user based on a deep learning algorithmto obtain an emotion classification model.

In this embodiment, for example, before the current facial feature ofthe user is obtained, the classification training processing isperformed on a plurality of biometric parameters of the user based on aconvolutional neural network deep learning algorithm to obtain anemotion classification model. For example, the classification trainingprocessing is performed on a plurality of facial features to obtain acorresponding emotion classification model of the user, such as a modelcorresponding to the emotion of anger.

In step S202, processing is performed on the biometric parameteraccording to the emotion classification model to determine the currentemotion type of the user.

For example, based on a pre-trained emotion classification modelcorresponding to, for example the user's angry emotion, the currentfacial feature of the user may be processed to determine the currentemotion type of the user. For example, the current emotion type isdetermined to be angry.

Further, in another exemplary embodiment of the present disclosure, thebiometric parameters include a plurality of biometric parameters indifferent classifications such as facial feature parameter and a soundfeature parameter. In the step S201, the step of performingclassification training processing on a plurality of biometricparameters based on a deep learning algorithm to obtain an emotionclassification model may include: for biometric parameters belonging toeach classification, performing the classification training processingon the biometric parameters belonging to the classification based on thedeep learning algorithm (such as a convolutional neural network deeplearning algorithm), so as to obtain a plurality of different emotionclassification sub-models correspondingly.

For example, the classification training processing can be performed ona large number of the user's facial features based on the convolutionalneural network deep learning algorithm to obtain for example facialemotion classification sub-models. Also, the classification trainingprocessing can be performed on a large number of the user's soundfeatures based on the convolutional neural network deep learningalgorithm to obtain, for example sound emotion classificationsub-models. Specifically, the deep learning-based algorithm may firstcollect a large amount of sample data, such as a face and sound featuresegments with emotion tags, and the tags may include, but are notlimited to, the following four tags: sadness, anger, happiness, andemotionlessness. Then, the convolutional neural network (CNN) deeplearning algorithm can be used to process and classify the facialfeatures to obtain a facial emotion classification sub-model. Similarly,the CNN deep learning algorithm can be used to process and classify thesound features to obtain a sound emotion classification sub-model.

In step S202, the step of performing processing on the biometricparameter according to the emotion classification model to determine thecurrent emotion type of the user includes: determining a classificationof the current biometric parameter, and selecting a correspondingemotion classification sub-model to process the current biometricparameter according to a classification determination result so as todetermine the current emotion type of the user.

For example, when it is determined that the current biometric parameteris the facial feature parameter, the facial feature parameter may beprocessed according to the facial emotion classification sub-model, soas to determine the current emotion type of the user. When it isdetermined that the current biometric parameter is the sound featureparameter, the sound feature parameter may be processed according to asound emotion classification sub-model, so as to determine the currentemotion type of the user. This allows targeted processing and improvesprocessing efficiency.

In step S102, an image of an artwork corresponding to the emotion typeis selected according to the determined current emotion type, and isrecommended to the user for display.

According to an exemplary embodiment, different images of thecorresponding artworks, such as the image of a paining can berecommended according to the determined current emotion type of theuser, as shown in FIGS. 4 and 5. For example, if the user's emotion isdepression, a paining which can give the user a positive mood, such as awork depicting an upward sunflower, can be recommended. If the user'semotion is angry, a paining which can give the user calm and peace, suchas a work depicting the blue sea, can be recommended to stabilize theuser's emotion. If the user's emotion is happy, the user is inclined tobuy things at this time, and a painting most likely to be purchased bythe user can be recommended according to the viewing history of theuser. The present disclosure is not limited to the above examples.

According to an exemplary embodiment of the present disclosure, themethod may further include the step of recommending the selected imageof the artwork to the user for display according to a preset timefrequency.

For example, the time frequency of recommending an artwork can be set tobe 3 days, etc., and the users can also reset the time frequencyaccording to their own preferences. A new painting can be recommended tothe user every 3 days, for example.

According to another exemplary embodiment of the present disclosure, themethod may further include the following step: recommending a relatedimage of an artwork according to the user's historical data to the user,when no emotion type is determined. For example, if the user's emotionis not detected, that is, when the emotion type is emotionless, therecommendation mode is used, and a related artwork is recommended to theuser according to the user's historical viewing data. The intelligentelectronic frame itself can have a recommendation system based oncontents of works, which can record the user's historical viewing data.

There may be following scenarios. If the user has multiple emotions in acertain period of time, how to recommend an artwork, or how to recommenda reasonable work to the user within the above-mentioned update periodof the works, such as 3 days, is determined. Based on the aboveembodiments, in another exemplary embodiment of the present disclosure,as shown in FIG. 3, the method may further include the following steps:

In step S301, when the user's current biometric parameters in differentclassifications are obtained, durations in which the user is indifferent emotion types determined according to the biometric parametersin each classification are recorded.

For example, according to the different emotion classificationsub-models, different current emotion types of the user are determined,the duration in which the user is in each emotion type is recorded. Thatis, the emotions are classified and the duration in which the user ineach emotion type is saved. For example, the duration of the angeremotion, the duration of the depression emotion and the duration of thehappy emotion are saved.

In step S302, the durations corresponding to the different emotion typesare compared, and an image of an artwork corresponding to an emotiontype which corresponds to a maximum one of the durations is selected andthe selected image of the artwork is recommended to the user fordisplay.

For example, if works shown on the electronic frame need to be updatednext time, the electronic frame can calculate the emotional energy ofeach emotion type according to the previous user's emotionclassification. It is assumed that the unit energy of various emotionsis equal, the emotional energy is simplified to accumulation of time.Therefore, based on the duration of each type of emotion describedabove, the emotion type of the user that has the longest duration, thatis, the emotion type having the greatest emotional energy may beselected, and a paining corresponding to the emotion having the greatestemotional energy is recommended to the user.

Specifically, emotional energies of four emotion types are determined,and the one with the greatest emotional energy is taken as therecommendation basis. For example, if the user's emotion is depression,a paining which can give the user a positive mood, such as a workdepicting an upward sunflower, can be recommended. If the user's emotionis angry, a paining which can give the user calm and peace, such as awork depicting the blue sea, can be recommended to stabilize the user'semotion. If the user's emotion is happy, the user is inclined to buythings at this time, and a painting most likely to be purchased by theuser can be recommended according to the viewing history of the user. Ifthe user's emotions are not detected, then the paining that the user ismost interested in is recommended according to the user's viewinghistory.

In the technical solutions according to embodiments of the presentdisclosure, emotion classification can be performed based on the deeplearning, and the emotion type of the user, such as anger, sadness,happy, etc. and be determined, thereby intelligently recommending areasonable image to the user based on different emotion types. Thetechnical solutions according to embodiments of the present disclosurecan understand the user better, so that the electronic frame such as theiGallery art frame can become friends of the user in daily life and areuser friendly. The electronic frame is not only an artwork, but also theexpert in emotional balance. The user has more than just an art frame,but also psychological healing and pleasure. In addition, the artisticupdate method of the intelligent electronic frame can be set to beautomatic and manual modes. If the manual mode is set, the user manuallyselects his/her favorite work. If an automatic mode is set, thesolutions according to the above exemplary embodiments can be used, andthe electronic frame can automatically push new works to the user everycertain time.

It should be noted that although various steps of methods of the presentdisclosure are described in a particular order in the figures, it is notrequired or implied that the steps must be performed in the specificorder, or all the steps shown must be performed to achieve the desiredresult. Additionally or alternatively, certain steps may be omitted,multiple steps may be combined into one step, and/or one step may bedecomposed into multiple steps and so on. In addition, it is also easyto understand that these steps may be performed synchronously orasynchronously in multiple modules/processes/threads, for example.

An exemplary embodiment provides an emotion recognition-based artworkrecommendation device. The device may be provided in, for example, anelectronic frame, and may be set in a processor of the electronic framein a form of instruction codes. Referring to FIG. 6, the device 600 mayinclude an emotion determination module 601 and an informationrecommendation module 602.

The emotion determination module 601 is configured to obtain a currentbiometric parameter of a user, and determine a current emotion type ofthe user according to the biometric parameter.

In an exemplary embodiment of the present disclosure, the biometricparameters may include, but are not limited to, facial featureparameters, or sound feature parameters. For example, the emotiondetermination module 601 can be the microphone and the camera providedon the electronic frame for collecting the biometric parameters such asthe sound feature parameters and the facial feature parameters of theuser, respectively. The emotion determination module 601 may also be asound sensor and a camera provided on the electronic frame forcollecting the biometric parameters such as the sound feature parametersand the facial feature parameters of the user, respectively.

The information recommendation module 602 is configured to select animage of an artwork corresponding to the emotion type according to thedetermined current emotion type, and recommend the image of the artworkto the user for display.

In an exemplary embodiment of the present disclosure, the device mayfurther include a classification training module 603 configured to,before the current biometric parameter of the user is obtained, performclassification training processing on a plurality of biometricparameters of the user based on a deep learning algorithm to obtain anemotion classification model. The emotion determination module 601 maybe further configured to perform processing on the biometric parameteraccording to the emotion classification model to determine the currentemotion type of the user.

In an exemplary embodiment of the present disclosure, the biometricparameters may include, but are not limited to, a plurality of biometricparameters in different classifications. The classification trainingmodule 603 may be further configured to, for biometric parametersbelonging to each classification, perform the classification trainingprocessing on the biometric parameters belonging to the classificationbased on the deep learning algorithm, so as to obtain a plurality ofdifferent emotion classification sub-models correspondingly. The emotiondetermination module 601 may be further configured to determine aclassification of the current biometric parameter, and select acorresponding emotion classification sub-model to process the currentbiometric parameter according to a classification determination result,so as to determine the current emotion type of the user.

In an exemplary embodiment of the present disclosure, the device mayfurther include an emotion type recording module 604 configured to, whenthe user's current biometric parameters in different classifications areobtained, record durations in which the user is in different emotiontypes determined according to the biometric parameters in eachclassification. The information recommendation module 602 may be furtherconfigured to compare the durations corresponding to the differentemotion types, and select an image of an artwork corresponding to anemotion type which corresponds to a maximum one of the durations andrecommend the image of the artwork to the user for display.

In the above-mentioned exemplary embodiments of the present disclosure,the biometric parameters in different classifications may include, butare not limited to, facial feature parameters and sound featureparameters.

In an exemplary embodiment of the present disclosure, the device mayfurther include a first artwork recommendation module configured torecommend the selected image of the artwork to the user for displayaccording to a preset time frequency.

In an exemplary embodiment of the present disclosure, the device mayfurther include a second artwork recommendation module configured torecommend a related image of an artwork according to the user'shistorical data to the user, when no emotion type is determined.

In an exemplary embodiment of the present disclosure, the deep learningalgorithm may include, but is not limited to, a convolutional neuralnetwork deep learning algorithm.

In an exemplary embodiment of the present disclosure, the device mayfurther include an identity verification module 605 configured to, afterthe user's current biometric parameter is obtained, verify validity ofan identity of the user according to the current biometric parameter.

Regarding the devices in the above embodiments, the specific manner inwhich each module performs operations has been described in detail inthe method embodiments, and will not be described in detail here.

It should be noted that although modules or units of devices forexecuting functions are described above, such division of modules orunits is not mandatory. In fact, features and functions of two or moreof the modules or units described above may be embodied in one module orunit in accordance with the embodiments of the present disclosure.Alternatively, the features and functions of one module or unitdescribed above may be further divided into multiple modules or units.The components displayed as modules or units may or may not be physicalunits, may be located in one place, or may be distributed over multiplenetwork units. Some or all of these modules can be selected to achievethe purpose of the present disclosure according to actual needs. Thoseof ordinary skill in the art can understand and implement the presentdisclosure without creative work.

In an exemplary embodiment of the present disclosure, there is alsoprovided a non-transitory computer-readable storage medium having storedthereon a computer program which, when executed by a processor, causesthe process to implement steps of the emotion recognition-based artworkrecommendation method described in any one of the above embodiments. Insome possible implementations, aspects of the present disclosure mayalso be implemented in the form of a program product, which includesprogram codes. When the program product runs on a terminal device suchas an electronic frame, the program codes are used to cause the terminaldevice to perform the steps according to various exemplary embodimentsof the present disclosure described in the emotion recognition-basedartwork recommendation methods of the present disclosure.

FIG. 7 shows a program product 700 for implementing the above methodaccording to an embodiment of the present disclosure. The programproduct may adopt a portable compact disc read-only memory (CD-ROM) andinclude program codes, and may be run on a terminal device, such as theelectronic frame. However, the program product of the present disclosureis not limited thereto. The readable storage medium may be any tangiblemedium containing or storing a program, and the program may be used byan instruction execution system, apparatus, or device or may be used incombination with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readablemedia. The readable medium may be a readable signal medium or a readablestorage medium. The readable storage medium may be, for example, but isnot limited to, an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system, apparatus, or device, or anycombination thereof. More specific examples (non-exhaustive list) of thereadable storage media include: an electrical connection with one ormore wires, a portable disk, a hard disk, a random access memory (RAM),a read-only memory (ROM), an erasable programmable read-only memory(EPROM or flash memory), an optical fiber, a portable compact discread-only memory (CD-ROM), an optical storage device, a magnetic storagedevice, or any suitable combination of the foregoing.

The computer-readable storage medium may include a data signal inbaseband or propagated as part of a carrier wave, in which a readableprogram code is carried. This propagated data signal can take manyforms, including but not limited to electromagnetic signals, opticalsignals, or any suitable combination of the foregoing. The readablestorage medium may also be any readable medium other than a readablestorage medium, and the readable medium may send, transfer, or transmita program for use by or in connection with an instruction executionsystem, apparatus, or device. The program code contained on the readablestorage medium may be transmitted using any appropriate medium,including but not limited to wireless, wired, optical fiber cable, RF,etc., or any suitable combination of the foregoing.

The program code for performing the operations of the present disclosuremay be written in any combination of one or more programming languages,which include object-oriented programming languages, such as Java, C++,etc. and also include conventional procedural programming language, suchas “C” or a similar programming language. The program code can beexecuted entirely on the user computing device, partly on the userdevice, executed as an independent software package, executed partly onthe user computing device and partly on the remote computing device, orexecuted entirely on the remote computing device or server. In the caseof a remote computing device, the remote computing device can beconnected to the user computing device through any kind of network,including a local area network (LAN) or a wide area network (WAN), ormay be connected to an external computing device (for example, using anInternet service provider to connect through the Internet).

In an exemplary embodiment of the present disclosure, there is alsoprovided an electronic device, which may include a processor, and amemory for storing instructions executable by the processor. Theprocessor is configured to execute the instructions to implement stepsof the emotion recognition-based artwork recommendation methods in anyone of the foregoing embodiments.

In an exemplary embodiment of the present disclosure, the electronicdevice may further include a body, a display part and a biometricfeature collection device. The body is, for example, a rectangular framebody. The display part is provided on the body. For example, the displaypart is embedded in the rectangular frame body, and configured todisplay an image of an artwork. The biometric feature collection deviceis also provided on the body, for example, on a frame of the rectangularframe body, and is configured to collect biometric feature parameters ofa user.

For example, the electronic device may be an electronic frame. Theartworks of different color styles can be stored in the memory inadvance, such as the images of the paintings. The display part mayinclude, but is not limited to, a display device such as an LED display,an LCD display, or an OLED display, which may display the images of theartworks, for example the images of the paintings. The biometriccollection device may include, but is not limited to, a microphone and acamera. The microphone and the camera may respectively collect theuser's sound features and the facial image features. In this way, afterthe biometric features are processed by the processor, paintings ofdifferent colors and styles can be selected and recommended to the useraccording to the different current emotions of the user.

Those skilled in the art can understand that various aspects of thepresent disclosure may be implemented as a system, a method, or aprogram product. Therefore, various aspects of the present disclosurecan be embodied in the following forms: a complete hardwareimplementation, a complete software implementation (including afirmware, a microcode, etc.), or a combination of the hardware and thesoftware, which can be collectively referred to as “circuit”, “module”or “system”.

An electronic device 800 according to this embodiment of the presentdisclosure is described below with reference to FIG. 8. The electronicdevice 800 shown in FIG. 8 is merely an example, and should not imposeany limitation on the functions and range of application of theembodiments of the present disclosure.

As shown in FIG. 8, the electronic device 800 is implemented in the formof a general-purpose computing device. The components of the electronicdevice 800 may include, but are not limited to, at least one processingunit 810, at least one storage unit 820, a bus 830 connecting differentsystem components (including the storage unit 820 and the processingunit 810), a display unit 840, and the like.

The storage unit stores program codes, and the program codes can beexecuted by the processing unit 810, so that the processing unit 810executes the steps of the emotion recognition-based artworkrecommendation methods according to various exemplary embodiments of thepresent disclosure. For example, the processing unit 810 may performsteps as shown in FIG. 1.

The storage unit 820 may include a readable medium in the form of avolatile storage unit, such as a random access storage unit (RAM) 8201and/or a cache storage unit 8202, and may further include a read-onlystorage unit (ROM) 8203.

The storage unit 820 may further include a program/utility tool 8204having a set of (at least one) program modules 8205. Such programmodules 8205 include, but are not limited to, an operating system, oneor more applications, other program modules, and program data. Each ofthese examples or some combination may include an implementation of thenetwork environment.

The bus 830 may be one or more of several types of bus structures,including a memory unit bus or a memory unit controller, a peripheralbus, a graphics acceleration port, a processing unit, or a local bususing any of a variety of bus structures.

The electronic device 800 may also communicate with one or more externaldevices 900 (such as a keyboard, a pointing device, a Bluetooth device,etc.), and may also communicate with one or more devices that enable theuser to interact with the electronic device 800, and/or communicate withany device (such as a router, a modem, etc.) that enables the electronicdevice 800 to communicate with one or more other computing devices. Thiscommunication can be performed through an input/output (I/O) interface850. Moreover, the electronic device 800 can also communicate with oneor more networks (such as a local area network (LAN), a wide areanetwork (WAN), and/or a public network, such as the Internet) throughthe network adapter 860. The network adapter 860 may communicate withother modules of the electronic device 800 through the bus 830. Itshould be understood that although not shown in the figure, otherhardware and/or software modules may be used in conjunction with theelectronic device 800, including but not limited to: microcode, a devicedriver, a redundant processing unit, an external disk drive array, aRAID system, a tape drive and a data backup storage system.

Through the description of the above embodiments, those skilled in theart will readily understand that the exemplary embodiments describedherein may be implemented by software or by a combination of softwarewith necessary hardware. Therefore, the technical solutions according toembodiments of the present disclosure may be embodied in the form of asoftware product, which may be stored in a non-volatile storage medium(which may be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) oron a network. A number of instructions are included to cause a computingdevice (which may be a personal computer, server) to perform the aboveemotion recognition-based artwork recommendation methods in accordancewith the embodiments of the present disclosure.

Other embodiments of the present disclosure will be apparent to thoseskilled in the art. The present application is intended to cover anyvariations, uses, or adaptations of the present disclosure, which are inaccordance with the general principles of the present disclosure andinclude common general knowledge or conventional technical means in theart that are not disclosed in the present disclosure. The specificationand embodiments are illustrative, and the real scope of the presentdisclosure is defined by the appended claims.

1. An emotion recognition-based artwork recommendation method, appliedin an electronic device having a display screen, wherein the methodcomprises: obtaining a current biometric parameter of a user;determining a current emotion type of the user according to the currentbiometric parameter; selecting an image of an artwork corresponding tothe current emotion type according to the determined current emotiontype; and recommending an image of the artwork to the user by displayingthe image of the artwork on the display screen.
 2. The emotionrecognition-based artwork recommendation method according to claim 1,wherein the method further comprises: before obtaining the currentbiometric parameter of the user, performing classification trainingprocessing on a plurality of biometric parameters of the user based on adeep learning algorithm to obtain an emotion classification model;wherein determining a current emotion type of the user according to thecurrent biometric parameter comprises: performing processing on thecurrent biometric parameter according to the emotion classificationmodel to determine the current emotion type of the user.
 3. The emotionrecognition-based artwork recommendation method according to claim 2,wherein the plurality of biometric parameters comprise a plurality ofbiometric parameters in different classifications; wherein performingclassification training processing on the plurality biometric parametersbased on the deep learning algorithm to obtain the emotionclassification model comprises: performing the classification trainingprocessing on the plurality of biometric parameters belonging to theclassification based on the deep learning algorithm, so as to obtain aplurality of different emotion classification sub-modelscorrespondingly; wherein performing processing on the current biometricparameter according to the emotion classification model to determine thecurrent emotion type of the user comprises: determining a classificationof the current biometric parameter, and selecting a correspondingemotion classification sub-model to process the current biometricparameter according to a classification determination result, so as todetermine the current emotion type of the user.
 4. The emotionrecognition-based artwork recommendation method according to claim 3,wherein the method further comprises: when a plurality of differentclassifications of current biometric parameters of a user are obtained,recording durations in which the user is in different emotion typesdetermined according to the biometric parameters in each classification;wherein selecting an image of an artwork corresponding to the currentemotion type and recommending the image of the artwork to the user bydisplaying the image of the artwork on the display screen comprises:comparing the durations corresponding to the different emotion types,and selecting an image of an artwork corresponding to an emotion typewhich corresponds to a maximum one of the durations and recommending theimage of the artwork to the user by displaying the image of the artworkon the display screen.
 5. The emotion recognition-based artworkrecommendation method according to claim 3, wherein the plurality ofbiometric parameters in different classifications comprise facialfeature parameters and sound feature parameters.
 6. The emotionrecognition-based artwork recommendation method according to claim 5,wherein the method further comprises: recommending a selected image ofthe artwork to the user for display according to a preset timefrequency.
 7. The emotion recognition-based artwork recommendationmethod according to claim 5, wherein the method further comprises: whenno emotion type is determined, recommending a related image of anartwork to the user according to historical data of the user.
 8. Theemotion recognition-based artwork recommendation method according toclaim 1, wherein the method further comprises: after obtaining thecurrent biometric parameter of the user, verifying validity of anidentity of the user according to the current biometric parameter.
 9. Anelectronic device, comprising: a body; a display device provided on thebody and configured to display an image of an artwork; a biometricfeature collection device provided on the body configured to collect abiometric feature parameter of a user; a hardware processor; a memoryfor storing instructions executable by the hardware processor, whereinthe hardware processor is configured to: determine a current emotiontype of the user according to a current biometric parameter obtained bythe biometric feature collection device; select an image of an artworkcorresponding to the current emotion type according to the currentemotion type; recommend the image of the artwork and cause the image ofthe artwork to be shown to the user in the display device.
 10. Theemotion recognition-based artwork recommendation device according toclaim 9, wherein the hardware processor is further configured to: beforethe current biometric parameter of the user is obtained, performclassification training processing on a plurality of biometricparameters of the user based on a deep learning algorithm to obtain anemotion classification model; perform processing on the currentbiometric parameter according to the emotion classification model todetermine the current emotion type of the user.
 11. The emotionrecognition-based artwork recommendation device according to claim 10,wherein the plurality of biometric parameters comprise a plurality ofbiometric parameters in different classifications; wherein the hardwareprocessor is further configured to perform the classification trainingprocessing on the plurality of biometric parameters belonging to theclassification based on the deep learning algorithm, so as to obtain aplurality of different emotion classification sub-modelscorrespondingly; wherein the hardware processor is further configured todetermine a classification of the current biometric parameter, andselect a corresponding emotion classification sub-model to process thecurrent biometric parameter according to a classification determinationresult, so as to determine the current emotion type of the user.
 12. Theemotion recognition-based artwork recommendation device according toclaim 11, wherein the hardware processor is further configured to: whena plurality of different classifications of current biometric parametersof the user are obtained, record durations in which the user is indifferent emotion types determined according to the biometric parametersin each classification; wherein the hardware processor is furtherconfigured to compare the durations corresponding to the differentemotion types, and select an image of an artwork corresponding to anemotion type which corresponds to a maximum one of the durations andrecommend the image of the artwork to the user by displaying the imageof the artwork on the display screen.
 13. The emotion recognition-basedartwork recommendation device according to claim 11, wherein theplurality of biometric parameters in different classifications comprisefacial feature parameters and sound feature parameters.
 14. The emotionrecognition-based artwork recommendation device according to claim 13,wherein the hardware processor is further configured to: recommend aselected image of the artwork to the user for display according to apreset time frequency.
 15. The emotion recognition-based artworkrecommendation device according to claim 13, wherein the hardwareprocessor is further configured to: when no emotion type is determined,recommend a related image of an artwork to the user according tohistorical data of the user.
 16. The emotion recognition-based artworkrecommendation device according to claim 9, wherein the furtherconfigured to: verify validity of an identity of the user according tothe current biometric parameter after the current biometric parameter ofthe user is obtained.
 17. A non-transitory computer-readable storagemedium having a computer program stored thereon, wherein, when theprogram is executed by a processor of an electronic device having adisplay screen, an emotion recognition-based artwork recommendationmethod is implemented, wherein the method comprises: obtaining a currentbiometric parameter of a user; determining a current emotion type of theuser according to the current biometric parameter; selecting an image ofan artwork corresponding to the current emotion type according to thecurrent emotion type; and recommending an image of the artwork to theuser by displaying the image of the artwork on the display screen. 18.(canceled)
 19. The computer-readable storage medium according to claim17, wherein the method further comprises: before obtaining the currentbiometric parameter of the user, performing classification trainingprocessing on a plurality of biometric parameters of the user based on adeep learning algorithm to obtain an emotion classification model; andwherein determining a current emotion type of the user according to thecurrent biometric parameter comprises: performing processing on thecurrent biometric parameter according to the emotion classificationmodel to determine the current emotion type of the user.
 20. Thecomputer-readable storage medium according to claim 19, wherein theplurality of biometric parameters comprise a plurality of biometricparameters in different classifications; wherein performingclassification training processing on the plurality of biometricparameters based on the deep learning algorithm to obtain an emotionclassification model comprises: performing the classification trainingprocessing on the plurality of biometric parameters belonging to theclassification based on the deep learning algorithm, so as to obtain aplurality of different emotion classification sub-modelscorrespondingly; wherein performing processing on the current biometricparameter according to the emotion classification model to determine thecurrent emotion type of the user comprises: determining a classificationof the current biometric parameter; and selecting a correspondingemotion classification sub-model to process the current biometricparameter according to a classification determination result, so as todetermine the current emotion type of the user.
 21. Thecomputer-readable storage medium according to claim 20, wherein themethod further comprises: when a plurality of different classificationsof current biometric parameters of a user are obtained, recordingdurations in which the user is in different emotion types determinedaccording to the biometric parameters in each classification; andwherein selecting the image of an artwork corresponding to the currentemotion type and recommending the image of the artwork to the user bydisplaying the image of the artwork on the display screen comprises:comparing the durations corresponding to the different emotion types;selecting an image of an artwork corresponding to an emotion type whichcorresponds to a maximum one of the durations; and recommending theimage of the artwork to the user by displaying the image of the artworkon the display screen.